Http adaptive streaming server, method, and client terminal based on network environment monitoring

ABSTRACT

The disclosure relates to a communication technique for combining a 5th generation (5G) communication system that supports higher data transmission rates after 4th generation (4G) systems with Internet of Things (IoT) technology and to system thereof. The disclosure can be applied for intelligent services based on 5G communication technology and IoT related technology (e.g., smart homes, smart buildings, smart cities, smart cars or connected cars, healthcare, digital education, retail businesses, security and safety related services, and the like). To do so, a method for providing, by a streaming server, HTTP adaptive streaming to a client terminal is provided. The method includes transmitting a first multimedia data to a terminal, monitoring a network environment between a streaming server and the terminal when the first multimedia data is transmitted to the terminal, and pushing at least one second multimedia data based on the network environment monitored.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is based on and claims priority under 35 U.S.C. §119(a) of Korean patent application number 10-2017-0056110, filed on May2, 2017, in the Korean Intellectual Property Office, the disclosure ofwhich is incorporated by reference herein in its entirety.

TECHNICAL FIELD

The disclosure relates to hypertext transfer protocol (HTTP) adaptivestreaming servers, methods, and client terminals.

BACKGROUND

In order to meet the demand for wireless data traffic that has increasedsignificantly since the 4th generation (4G) communication system came tothe market, there are ongoing efforts to develop enhanced 5th generation(5G) communication systems or pre-5G communication systems. For theseand other reasons, the 5G communication system or pre-5G communicationsystem is typically called the beyond 4G network communication system orpost long term evolution (LTE) system.

For higher data transmit rates, 5G communication systems are to beimplemented on ultra high frequency bands (mmWave), such as, e.g., 60GHz. To mitigate pathloss on the ultra high frequency band and increasethe reach of radio waves, the following techniques are being taken intoaccount for the 5G communication system: beamforming, massivemulti-input multi-output (MIMO), full dimensional MIMO (FD-MIMO), arrayantenna, analog beamforming, and large-scale antenna.

Also being developed are various technologies for the 5G communicationsystem to have an enhanced network, such as evolved or advanced smallcell, cloud radio access network (cloud RAN), ultra-dense network,device-to-device (D2D) communication, wireless backhaul, moving network,cooperative communication, coordinated multi-point (CoMP), andinterference cancellation.

There are also other various schemes under development for the 5G systemincluding, e.g., hybrid frequency-shift keying (FSK) and quadratureamplitude modulation (QAM) modulation (FQAM) and sliding windowsuperposition coding (SWSC), which are advanced coding modulation (ACM)schemes, and filter bank multi-carrier (FBMC), non-orthogonal multipleaccess (NOMA) and sparse code multiple access (SCMA), which are advancedaccess schemes.

Meanwhile, the Internet is evolving from a human-centered connectionnetwork by which humans create and consume information, into an Internetof Things (IoT) network by which information is communicated andprocessed between things or other distributed components. The Internetof Everything (IoE) technology is an example of a combination of bigdata processing technology and the IoT technology through, e.g., aconnection with a cloud server.

To implement the IoT, technology elements, such as a sensing technology,wired/wireless communication and network infra, service interfacetechnology, and security technology, are required. There is also recentongoing research for other inter-object connection technologies, such asa sensor network, machine-to-machine (M2M), or machine-typecommunication (MTC).

In the IoT environment, intelligent Internet technology (IT) servicesmay be offered that collect and analyze data generated by thingsconnected with one another to create a new value. The IoT may havevarious applications, such as a smart home, a smart building, a smartcity, a smart car or a connected car, a smart grid, health-care, smartappliance industry, or state-of-the-art medical services, throughconversion or integration of existing IT technologies and variousindustries.

Thus, there are various ongoing efforts to apply the 5G communicationsystem to the IoT network. For example, the sensor network, M2M, machinetype communication (MTC), or other 5G techniques are being implementedby schemes, such as beamforming, MIMO, and array antenna schemes. Theabove-mentioned application of the cloud radio access network as a bigdata processing technique may also be said to be an example of theconvergence of 5G and IoT technologies.

Adaptive streaming is a technique for adaptively selecting an adequatevideo quality considering the environment of the network and clientterminal, which is required to meet the quality that users experience ina varied wireless network environment. Such adaptive streaming isapplied based on various network protocols, such as the real-timetransport protocol (RTP) or real-time streaming protocol (RTSP).Recently attracting attention are hypertext transfer protocol (HTTP)adaptive streaming techniques using the HTTP.

HTTP adaptive streaming may adopt existing HTTP technology, advancingnetwork compatibility for streaming services and advantageously allowingfor reuse of existing HTTP services for streaming services. Further,HTTP adaptive streaming puts the core of streaming controls with theclient, thus being able to provide streaming services to clients toreceive services.

FIG. 1 is a view illustrating an example of communication between aclient and a server in an HTTP adaptive streaming technique according tothe related art.

In the HTTP adaptive streaming technique, a client terminal receives onedata segment (e.g., response_1 of FIG. 1) from a streaming server eachtime the client terminal sends a request (e.g., request_1 of FIG. 1) forone data segment. Thus, the client terminal needs to send out as manyrequest messages as the number of data segments, causing an accumulationof request messages over time and hence overloading the client terminal,streaming server, and the network.

The above information is presented as background information only, andto assist with an understanding of the disclosure. No determination hasbeen made, and no assertion is made, as to whether any of the abovemight be applicable as prior art with regard to the disclosure.

SUMMARY

Aspects of the disclosure are to address at least the above-mentionedproblems and/or disadvantages, and to provide at least the advantagesdescribed below. Accordingly, an aspect of the disclosure is to providea streaming server, a client terminal, a streaming system, and a methodusing a hypertext transfer protocol (HTTP) adaptive streaming schemebased on network environment monitoring.

Another aspect of the disclosure is to provide a scheme for relieving aclient terminal, a streaming server, and a streaming system of overloadin running HTTP adaptive streaming.

Another aspect of the disclosure is to provide a scheme for providingstreaming data that adaptively varies depending on variations in thenetwork environment.

In accordance with another aspect of the disclosure, a method forproviding, by a streaming server, HTTP adaptive streaming to a clientterminal is provided. The method includes transmitting a firstmultimedia data item to the client terminal, monitoring a networkenvironment between the streaming server and the client terminal whenthe first multimedia data item is transmitted to the client terminal,and pushing at least one second multimedia data item based on themonitored network environment.

In accordance with another aspect of the disclosure, pushing may includefurther transmitting the at least one second multimedia data item to theclient terminal regardless of whether there is a request from the clientterminal after the streaming server transmits the first multimedia dataitem to the client terminal.

In accordance with another aspect of the disclosure, pushing may includefurther transmitting the at least one second multimedia data item to theclient terminal at predetermined times.

In accordance with another aspect of the disclosure, the predeterminedtimes may be determined based on the first multimedia data item, thenumber of the at least one second multimedia data item, or a playingduration.

In accordance with another aspect of the disclosure, the networkenvironment may be a transmission rate between the streaming server andthe client terminal. Monitoring may include calculating the transmissionrate based on a difference between a time of starting transmission ofthe first multimedia data item and a time of ending the transmission ofthe first multimedia data item.

In accordance with another aspect of the disclosure, the difference maybe determined as a difference between the time of starting thetransmission of the first multimedia data item and a time of receivingan acknowledgment (ACK) signal for the first multimedia data itemtransmitted.

In accordance with another aspect of the disclosure, the firstmultimedia data item may include video data or audio data split intosegments.

In accordance with another aspect of the disclosure, pushing may includefurther transmitting the at least one second multimedia data item with aresolution different from a resolution of the first multimedia dataitem.

In accordance with another aspect of the disclosure, pushing may includefurther transmitting the at least one second multimedia data item with aresolution lower than the resolution of the first multimedia data itemwhen the calculated transmission rate is lower than a prior transmissionrate.

In accordance with another aspect of the disclosure, pushing may includefurther transmitting the at least one second multimedia data item with aresolution lower than the resolution of the first multimedia data itemwhen the calculated transmission rate is lower than a transmission ratecorresponding to the resolution of the first multimedia data item.

In accordance with another aspect of the disclosure, a streaming serverconfigured to provide HTTP adaptive streaming to a client terminal isprovided. The streaming server includes a server communicator configuredto transmit a first multimedia data item to the client terminal, and aserver controller configured to control to monitor a network environmentbetween the streaming server and the client terminal when the firstmultimedia data item is transmitted to the client terminal and push atleast one second multimedia data item based on the network environmentbeing monitored.

In accordance with another aspect of the disclosure, pushing may includefurther transmitting the at least one second multimedia data item to theclient terminal regardless of whether there is a request from the clientterminal after the streaming server transmits the first multimedia dataitem to the client terminal.

In accordance with another aspect of the disclosure, the servercontroller may be configured to further transmit the at least one secondmultimedia data item to the client terminal at predetermined times.

In accordance with another aspect of the disclosure, the predeterminedtimes may be determined based on the first multimedia data item, thenumber of the at least one second multimedia data item, or a playingduration.

In accordance with another aspect of the disclosure, the networkenvironment may be a transmission rate between the streaming server andthe client terminal.

In accordance with another aspect of the disclosure, the servercontroller may be configured to calculate the transmission rate based ona difference between a time of starting transmission of the firstmultimedia data item and a time of ending the transmission of the firstmultimedia data item.

In accordance with another aspect of the disclosure, the difference maybe determined as a difference between the time of starting thetransmission of the first multimedia data item and a time of receivingan ACK signal for the first multimedia data item transmitted.

In accordance with another aspect of the disclosure, the firstmultimedia data item may include video data or audio data split intosegments.

In accordance with another aspect of the disclosure, the servercontroller may be configured to push the at least one second multimediadata item with a resolution different from a resolution of the firstmultimedia data item.

In accordance with another aspect of the disclosure, the servercontroller may be configured to push the at least one second multimediadata item with a resolution lower than the resolution of the firstmultimedia data item when the calculated transmission rate is lower thana prior transmission rate.

In accordance with another aspect of the disclosure, the servercontroller may be configured to push the at least one second multimediadata item with a resolution lower than the resolution of the firstmultimedia data item when the calculated transmission rate is lower thana transmission rate corresponding to the resolution of the firstmultimedia data item.

In accordance with another aspect of the disclosure, a client terminalconfigured to receive HTTP adaptive streaming from a streaming server isprovided. The client terminal includes a terminal communicatorconfigured to receive a first multimedia data item from the streamingserver and a terminal controller configured to calculate a parametervalue regarding a network environment between the streaming server andthe client terminal when the first multimedia data item is transmittedto the terminal communicator and send a request for at least one secondmultimedia data item to be received by pushing to the streaming serverbased on the calculated parameter value.

In accordance with another aspect of the disclosure, the parameter valueregarding the network environment may be calculated based on adifference between a time when the first multimedia data item istransmitted from the streaming server and a time when the firstmultimedia data item is received by the client terminal.

Other aspects, advantages, and salient features of the disclosure willbecome apparent to those skilled in the art from the following detaileddescription, which, taken in conjunction with the annexed drawings,discloses various embodiments of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainembodiments of the disclosure will be more apparent from the followingdescription taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 is a view illustrating an example of communication between aclient and a server in a hypertext transfer protocol (HTTP) adaptivestreaming technique according to the related art;

FIG. 2 is a view illustrating a streaming system to provide HTTPadaptive streaming by monitoring a network environment according to anembodiment of the disclosure;

FIG. 3 is a flowchart illustrating a method for providing HTTP adaptivestreaming by monitoring a network environment according to an embodimentof the disclosure;

FIG. 4 is a view illustrating an example of a method for providing HTTPadaptive streaming by monitoring a network environment as shown in FIG.3, according to an embodiment of the disclosure;

FIG. 5 is a flowchart illustrating a method for providing HTTP adaptivestreaming by monitoring a network environment according to an embodimentof the disclosure;

FIG. 6 is a view illustrating an example of a method for providing HTTPadaptive streaming by monitoring a network environment as shown in FIG.5, according to an embodiment of the disclosure;

FIG. 7 is a flowchart illustrating a method for providing HTTP adaptivestreaming by monitoring a network environment according to an embodimentof the disclosure;

FIG. 8 is a block diagram illustrating a streaming server according toan embodiment of the disclosure;

FIG. 9 is a block diagram illustrating a client terminal according to anembodiment of the disclosure; and

FIG. 10 is a block diagram illustrating a streaming system according toan embodiment of the disclosure.

Throughout the drawings, like reference numerals will be understood torefer to like parts, components, and structures.

DETAILED DESCRIPTION

The following description with reference to the accompanying drawings isprovided to assist in a comprehensive understanding of variousembodiments of the disclosure as defined by the claims and theirequivalents. It includes various specific details to assist in thatunderstanding, but these are to be regarded as merely exemplary.Accordingly, those of ordinary skill in the art will recognize thatvarious changes and modifications of the various embodiments describedherein can be made without departing from the scope and spirit of thedisclosure. In addition, descriptions of well-known functions andconstructions may be omitted for clarity and conciseness.

The terms and words used in the following description and claims are notlimited to the bibliographical meanings, but are merely used to enable aclear and consistent understanding of the disclosure. Accordingly, itshould be apparent to those skilled in the art that the followingdescription of various embodiments of the disclosure is provided forillustration purpose only, and not for the purpose of limiting thedisclosure as defined by the appended claims and their equivalents.

The terms “first” and “second” as used herein may be used to describevarious components, but the components should not be limited by theterms. The terms are used only to distinguish one component fromanother. When an element “includes” another element, the element mayfurther include the other element, rather excluding the other element,unless particularly stated otherwise. As used herein, the term “unit”means a software element or a hardware element such as afield-programmable gate array (FPGA) or an application specificintegrated circuit (ASIC). A unit plays a certain role. However, theterm “unit” is not limited as meaning a software or hardware element. A“unit” may be configured in a storage medium that may be addressed ormay be configured to reproduce one or more processors.

It is to be understood that the singular forms “a,” “an,” and “the”include plural referents unless the context clearly dictates otherwise.Thus, for example, reference to “a component surface” includes referenceto one or more of such surfaces.

Accordingly, as an example, a “unit” includes elements, such as softwareelements, object-oriented software elements, class elements, and taskelements, processes, functions, attributes, procedures, subroutines,data segments of program codes, drivers, firmware, microcodes, circuits,data, databases, data architectures, tables, arrays, and variables. Afunction provided in an element or a “unit” may be combined withadditional elements or may be split into sub elements or sub units.

As used herein, “transmission rate” may also be represented as a datatransmission rate, bitrate, or transfer rate, which may mean the amountof information transmitted per unit time, the speed at which data istransmitted, or the number of bits (bits/sec or bps) processed perparticular time unit (e.g., second).

According to the present disclosure, there are proposed a streamingserver, a client terminal, a streaming system, and a method forproviding hypertext transfer protocol (HTTP) adaptive streaming based onthe results of monitoring the network environment between a streamingserver and a client terminal.

FIG. 2 is a view illustrating a streaming system to provide HTTPadaptive streaming by monitoring a network environment according to anembodiment of the disclosure.

Referring to FIG. 2, a streaming system 200 includes a streaming server220 and a client terminal 210. The streaming server 220 is an entity toprovide media data, e.g., video, over the network by communicating withthe client terminal 210. The client terminal 210 is an entity to receivemedia data, e.g., video, by communicating with the streaming server 220,and which may also be referred to as a terminal, mobile terminal, userequipment (UE), mobile station (MS), mobile equipment (ME), or device.

The streaming server 220 may store multimedia data that may be providedto the client terminal 210. The multimedia data may include video dataor audio data. Further, the multimedia data may also denote a datasegment described below. The streaming server 220 may temporarily storemultimedia data obtained through wired/wireless communication withsources that are outside of the streaming server 220.

The streaming server 220 may split the multimedia data into a pluralityof data segments and store the data segments. The data segments may alsobe referred to as “chunks”. The plurality of data segments stored in thestreaming server 220 may be subject to different conditions for, e.g.,playing duration (e.g., 1 second, 10 seconds, or 1 minute), playingtiming (e.g., 00:04:30 or 01:42:15), resolution (e.g., 480p, 720p, or1080p or high, med, or low), sound quality (e.g., 129 kbps, 192 kbps, or320 kbps), or coding scheme (e.g., MPEG-TS, MPEG-4 ASP, or H.264/MPEG-4AVC). Further, the streaming server 220 may store the file name, segmentserial number, playing timing, playing duration, resolution, or otherinformation about each data segment, with such pieces of informationcorresponding to each data segment.

The client terminal 210 may produce a control environment, e.g., a userinterface (UI), that enables the selection of multimedia data to beplayed. The UI that the client terminal 210 produces may provide forexample, a list of multimedia data in the streaming server 220, a windowfor selecting a resolution of video data, and a window for selecting asound quality of audio data. The UI may be provided to the user of theclient terminal 210 through a screen of the client terminal 210.

Upon receiving a selection of multimedia data through the UI, the clientterminal 210 may send a request for transmitting the selected multimediadata to the streaming server 220. The request of the client terminal 210may include information about multimedia data that the client terminal210 is to receive. The information about the multimedia data may includethe name, data segments, and bitrate of the multimedia data. When thestreaming server 220 delivers a corresponding image throughwired/wireless communication, the client terminal 210 may display thereceived image. The UI which the client terminal 210 executes may alsoinclude a function of providing a control environment to control theplaying timing, resolution, or sound quality of the multimedia data thatis playing or to play new multimedia data.

The client terminal 210 and the streaming server 220 may communicatewith each other through wired/wireless communication. The wiredcommunication may include, e.g., universal serial bus (USB), highdefinition multimedia interface (HDMI), recommended standard (RS)-232,or plain old telephone service (POTS). The wireless communication mayinclude, e.g., Bluetooth communication, Bluetooth low energy (BLE)communication, near-field communication (NFC), wireless local areanetwork (WLAN) communication, ZigBee communication, infrared dataassociation (IrDA) communication, wireless-fidelity (Wi-Fi) direct (WFD)communication, ultra-wideband (UWB) communication, Ant+ communication,Wi-Fi communication, 3rd generation (3G), 4th generation (4G), and 5thgeneration (5G) communication.

Upon receipt of a request for executing multimedia data through the UI,the streaming server 220 may transmit a predetermined number of datasegments to the client terminal 210. The predetermined number may bepreviously determined by the user or processor, arbitrarily or as per acriterion, or based on the network environment between the streamingserver 220 and the client terminal 210.

The streaming server 220 may select a data segment to be transmittedbased on the network environment between the streaming server 220 andthe client terminal 210 and transmit the selected data segment. This isdescribed below in detail with reference to FIGS. 3 to 6.

FIG. 3 is a flowchart illustrating a method for providing HTTP adaptivestreaming by monitoring a network environment according to an embodimentof the disclosure.

Referring to FIG. 3, upon receipt of a request for multimedia data fromthe client terminal, the streaming server transmits a predeterminednumber of data segments (e.g., “n” data segments) to the client terminalat operation 310. The predetermined number “n” may be set by the user ordeveloper of the client terminal or the controller (or at least oneprocessor) of the streaming server.

When a predetermined number “n” of data segments are transmitted to theclient terminal, the streaming server calculates, e.g., monitors,parameter values regarding the network environment between the streamingserver and the client terminal based on, e.g., a total size (e.g., totaldata volume) of the “n” data segments transmitted from the streamingserver to the client terminal, and the time for the streaming server totransmit the “n” data segments at operation 320.

For example, the streaming server may calculate a mean transmission rate(e.g., a parameter value regarding the network environment) for eachdata segment using Equation 1 below:

$\begin{matrix}{{{Mean}\mspace{14mu} {transmission}\mspace{14mu} {rate}} = {\frac{{Total}\mspace{14mu} {size}\mspace{14mu} {of}\mspace{14mu} {data}\mspace{14mu} {segments}\mspace{14mu} {transmitted}}{{Time}\mspace{14mu} {taken}\mspace{14mu} {to}\mspace{14mu} {transmit}\mspace{14mu} n\mspace{14mu} {data}\mspace{14mu} {segments}} \times \frac{1}{n}}} & {{Equation}\mspace{14mu} 1}\end{matrix}$

The streaming server may select a resolution of a data segment to bepushed to the client terminal based on the mean transmission ratecalculated using Equation 1, and pushes at least one data segment havingthe selected resolution to the client terminal at operation 330. “Push”may mean that the streaming server automatically or periodicallytransmits data to the client terminal even with no additional requestfrom the client terminal after the streaming server's first transmissionto the client terminal in response to the request for data transmissionfrom the client terminal to the streaming server.

As an example, where the mean transmission rate calculated usingEquation 1 is lower than a prior (or first) transmission rate, thestreaming server may push a data segment having a resolution lower thanthat of the data segment already transmitted to the client terminal. Thereason to push the data segment having a lower resolution than that ofthe data segment previously sent is that continuing to push datasegments having the same resolution despite a reduced transmission ratebetween the streaming server and the client terminal as compared withbefore, may cause a delay or buffering when the client terminal playsvideo data.

As another example, where the mean transmission rate calculated usingEquation 1 is lower than a minimum mean transmission rate required forthe client terminal to play, without delay, video data with apredetermined resolution, the streaming server may transmit to theclient terminal the data segment regarding the video data having thelower resolution than the predetermined resolution.

Even where the streaming server is transmitting data segments to theclient terminal, if the client terminal requests the streaming server tostop transmitting data segments, the streaming server may stop thetransmission of data segments immediately or with a time gap regardlessof the current proceeding step (or transmission step).

By the above-described streaming method, the streaming server mayprovide delay-free or buffering-free streaming services to the user bytransmitting data segments adequate or appropriate for the actualnetwork environment.

FIG. 4 is a view illustrating an example of a method for providing HTTPadaptive streaming by monitoring a network environment as shown in FIG.3, according to an embodiment of the disclosure.

Referring to FIGS. 3 and 4, upon receipt of “n” requests (requests 1 ton of FIG. 4) for transmission of data segments from the client terminal,the streaming server transmits “n” data segments to the client terminal.The streaming server may transmit all of the “n” data segments (seg_1 toseg_n of FIG. 4) to the client terminal and calculate parameter valuesregarding the network environment between the streaming server and theclient terminal while transmitting the “n” data segments to the clientterminal, based on the size of the “n” data segments and the time totransmit the “n” data segments. The parameter values may be meantransmission rates calculated using Equation 1 above, and can berepeated for “m” requests.

As an example, where the mean transmission rate calculated usingEquation 1 is lower than a prior transmission rate (e.g., the meantransmission rate between the streaming server and the client terminalupon transmitting seg_1 to seg_n), the streaming server may push a datasegment having a resolution lower than that of the data segment alreadytransmitted to the client terminal.

As another example, where the mean transmission rate that the streamingserver calculated using Equation 1 above is lower than the transmissionrate corresponding to the resolution of the data segment previouslytransmitted (e.g., a minimum transmission rate required for the videodata corresponding to the previously transmitted segment to be played onthe client terminal without delay), the streaming server may push datasegments having a resolution lower than the data segment previouslytransmitted to the client terminal. For example, where the previouslytransmitted data segment has a resolution of 1080p (under the assumptionthat the transmission rate should be higher than a minimum of 3 Mpbs toplay, without delay, videos with a resolution of 1080p), if the meantransmission rate calculated by Equation 1 is 2 Mbps, the streamingserver may transmit data segments having a lower resolution (e.g., 480por 720p) than the resolution (1080p) of the previously transmitted datasegment to the client terminal.

As another example, where the mean transmission rate calculated usingEquation 1 is higher than the prior (or before pushing) transmissionrate or higher than the transmission rate corresponding to theresolution of the pre-transmitted data segment, the streaming server maykeep the resolution of data segments to be pushed identical to theresolution of the pre-transmitted data segment, or as per apredetermined criterion or the user's selection entered to the clientterminal, may transmit data segments having a higher resolution than thepre-transmitted data segment to the client terminal.

The scheme described in connection with FIG. 1 may disadvantageouslyoverload the system because of the need for transmitting “n” requests toobtain “n” data segments. Yet, the method described above in connectionwith FIGS. 3 and 4 enables a plurality of segments to be pushed at onerequest, advantageously relieving the system of load.

FIG. 5 is a flowchart illustrating a method for providing HTTP adaptivestreaming by monitoring a network environment according to an embodimentof the disclosure.

Referring to FIG. 5, the streaming server receives an initial requestthat the client terminal produces, from the client terminal based on theuser's selection received through the UI at operation 510. The initialrequest may be, e.g., an HTTP request message. The initial request maycontain at least one of a multimedia data name (video file name),bitrate, and segment serial number.

The streaming server produces an initial response corresponding to theinitial request received from the client terminal and transmits theproduced initial response to the client terminal at operation 520. Here,the initial response may be, e.g., an HTTP response message. The initialresponse may contain any one of the data segments of the multimedia datathat the user has selected through the UI of the client terminal.

When the initial response reaches the client terminal, the clientterminal produces an acknowledgment (ACK) signal and transmits the ACKsignal to the streaming server at operation 530. The ACK signal may bean ACK code to notify that the initial response has arrived at theclient terminal.

The streaming server calculates a parameter value, e.g., a transmissionrate, regarding the network environment between the streaming server andthe client terminal based on the size of the data segment transmitted tothe client terminal and the time taken to transmit the data segment tothe client terminal at operation 540. As an example, the streamingserver may determine the network environment based on the differencebetween the time when the streaming server starts to transmit the datasegment and the time when the ACK signal from the client terminalarrives at the streaming server. As a specific calculation scheme,Equation 2 may be used which stems from Equation 1. Equation 2represents Equation 1 where the predetermined number “n” is 1.

$\begin{matrix}{{{Transmission}\mspace{14mu} {rate}} = \frac{{Size}\mspace{14mu} {of}\mspace{14mu} {data}\mspace{14mu} {segment}\mspace{14mu} {transmitted}}{\begin{matrix}{{{Reception}\mspace{14mu} {time}\mspace{14mu} {of}\mspace{14mu} {ACK}} -} \\{{Start}\mspace{14mu} {time}\mspace{14mu} {of}\mspace{14mu} {data}\mspace{14mu} {transmission}}\end{matrix}}} & {{Equation}\mspace{14mu} 2}\end{matrix}$

Since the streaming server calculates the parameter value (e.g., datatransmission rate) regarding the network environment based on the timewhen the ACK signal from the client terminal is received by thestreaming server, the streaming server may obtain a monitoring resultprecisely reflecting the environment in which the user actually receivessupport for streaming services.

The streaming server may push data segments of different states based onthe parameter value (e.g., transmission rate) calculated using Equation2 at operation 550. For example, where the transmission rate calculatedusing Equation 2 is lower than a prior transmission rate, the streamingserver may push a data segment having a resolution lower than that ofthe data segment already transmitted to the client terminal. Theresolution of data segments that the streaming server is to transmit tothe client terminal may be varied stepwise or depending on thedifferences between the transmission rate calculated by Equation 2 andprior transmission rates.

As set forth above, in the method described in connection with FIG. 5the network environment is monitored by the streaming server, not by theclient terminal and therefore, the client terminal is less likely to beoverloaded, enabling steady monitoring of the network environment. Theclient terminal, which is a common mobile terminal, e.g., a cellularphone, has limited battery life and CPU resources, requiring that thebattery and CPU resources be used in an efficient way. The methoddisclosed in FIG. 5 may reduce resource consumption on the clientterminal (e.g., a mobile terminal) by letting the streaming servermonitor the network environment, allowing for efficient use of theresources of the client terminal.

The monitoring method described in connection with FIG. 5 and Equation 2differs from the monitoring method described with reference with FIGS. 3and 4 and Equation 1 as follows.

The monitoring method set forth with reference to FIGS. 3 and 4 andEquation 1 monitors the network environment after all of the “n”segments have been transmitted to the client terminal, and pushes otherdata segments based on a result thereof. In this case, before thetransmission of all of the “n” data segments, the same data segments areforced to be transmitted despite a variation in the network environment,causing the waste of network bandwidth and resources of the streamingserver and client terminal. However, the monitoring method describedwith reference to FIG. 5 and Equation 2 performs continuous monitoringof the network environment using an ACK corresponding to one push orinitial response. This prevents the unnecessary waste of bandwidth ortransmission resources that would result from the monitoring method ofFIGS. 3 and 4 and Equation 1.

Where many network nodes are present between the streaming server andthe client terminal and each network node retains many buffers, the meantransmission rate calculated using Equation 1 may be measured to behigher than the actual mean transmission rate. In this case, the usercannot receive the corresponding video data at proper speed, and a delayor buffering may occur in the video data. However, the monitoring methodof FIG. 5 and Equation 2 performs monitoring while precisely reflectingthe environment in which the user actually receives support forstreaming services, thus avoiding or reducing a delay or buffering inthe streaming video.

The method set forth with reference to FIGS. 3 and 4 requires that theclient terminal send out a predetermined number of requests for datasegments. However, the method described in connection with FIG. 5differs in that only one segment request (e.g., an initial request) isneeded.

FIG. 6 is a view illustrating an example of a method for providing HTTPadaptive streaming by monitoring a network environment as shown in FIG.5, according to an embodiment of the disclosure.

Referring to FIG. 6, when the client terminal receives a selection formultimedia data from the user through its UI, the client terminal mayproduce an initial request (request of FIG. 6) and transmit the initialrequest to the streaming server. Upon receipt of the initial requestfrom the client terminal, the streaming server produces an initialresponse (response of FIG. 6) and transmits the initial response to theclient terminal. When the initial response arrives at the clientterminal, the client terminal may produce an ACK signal and transmit theACK signal to the streaming server.

The streaming server may calculate the transmission rate using Equation2 above, based on at least one of a size of the data segment transmittedto the client terminal or a time taken to transmit the data segment tothe client terminal.

The streaming server may determine the state of the data segment to bepushed to the client terminal based on the calculated transmission rateand push the data segment corresponding to the determined state to theclient terminal. Referring to FIG. 6, the streaming server may calculatethe transmission rate when the streaming server transmits the responseto the client terminal by applying Equation 2 and the difference betweenthe time when the response is transmitted to the client terminal and thetime of receipt of ACK_1, determine the state of the data segment to betransmitted as push_1 based on the calculated transmission rate, andpush the data segment, as push_1, corresponding to the determined stateto the client terminal. This rule may also be applicable to push_2 andpush_(n−1) of FIG. 6.

Additionally, the streaming server may push data segments to the clientterminal at each predetermined cycle using a timer (execute timer ofFIG. 6). The streaming server may also transmit additional responses ateach cycle using the timer and monitor the network environment (networkmonitoring of FIG. 6) using the ACK signal that the client terminalsends. The predetermined cycle may be set based on at least one thenumber and length of data segments.

FIG. 7 is a flowchart illustrating a method for providing HTTP adaptivestreaming by monitoring a network environment according to an embodimentof the disclosure.

Referring to FIG. 7, upon entry of multimedia data to be executedthrough the UI of the client terminal by the user, the client terminalsends a request for a data segment regarding the multimedia data to thestreaming server at operation 710.

The client terminal receives the data segment that it requested from thestreaming server at operation 720. Here, the data segment that thestreaming server transmits, along with the time information about thetime that the data segment is transmitted from the streaming server, maybe transmitted to the client terminal.

During the course when the client terminal receives the data segmentfrom the streaming server, the client terminal calculates a parametervalue regarding the network environment between the client terminal andthe streaming server at operation 730. Specifically, the client terminalmay calculate the parameter value, e.g., data transmission rate,regarding the network environment between the client terminal and thestreaming server using the size of the data segment transmitted and thedifference between the time when the data segment is transmitted fromthe streaming server and the time when the data segment arrives at theclient terminal.

The client terminal requests an additional data segment based on thecalculated parameter value regarding the network environment atoperation 740. For example, upon determining that the calculated datatransmission rate is high, the client terminal may send a request forvideo data with a higher resolution, and upon determining that thecalculated data transmission rate is low, the client terminal may send arequest for video data with a lower resolution.

The client terminal may receive data segments from the streaming server.The data segment that the streaming server sends to the client terminalmay be the one that the client terminal has requested the streamingserver to transmit. The client terminal reconfigures and decodes thereceived data segment and plays the multimedia data that the user hasselected at operation 750.

FIG. 8 is a block diagram illustrating a streaming server according toan embodiment of the disclosure.

Referring to FIG. 8, a streaming server 800 may include at least one ofa sever controller 810 (or at least one processor) and a servercommunication unit 820 (or communicator or transceiver). The servercontroller 810 may include a network monitoring unit 830.

The streaming server 800 may further include a storage unit (not shown)for storing a plurality of multimedia data received from, and to betransmitted to a client terminal. The multimedia data may include atleast one of video data and audio data.

The server controller 810 may split the multimedia data stored in thestorage unit into a plurality of data segments. The plurality datasegments split by the server controller 810 may be stored in the storageunit. As an example, the server controller 810 may split the video datastored in the storage unit at predetermined time intervals (e.g., every10 seconds), compress them into a predetermined format (e.g., MPEG-TS)by an encoding process, and store them in the storage unit.

The streaming server 800 may still further include a temporary storageunit for temporarily storing and managing at least one of video data andaudio data received from sources outside of the streaming server.Further, the server controller 810 may split multimedia data, which isobtained from sources outside of the streaming server 800 andtemporarily stored in the temporary storage unit, into a plurality ofdata segments. For example, the server controller 810 may split onevideo data item into a plurality of data segments to which differentconditions are applied for at least one of playing duration, playingtiming, resolution, sound quality, or coding scheme.

The storage unit may store the plurality of data segments along withtheir respective data file names, serial numbers, start times, orlengths or one or more combinations thereof.

The server communication unit 820 is connected with the client terminalvia wired/wireless communication. The server communication unit 820 mayreceive the request signal for multimedia data from the client terminaland transmit a plurality of data segments, which have been determined tobe transmitted to the server controller 820, to the client terminal.

The server controller 810 may select a data segment based on, e.g., thetype, time, image quality, or sound quality of multimedia data (e.g.,video data or audio data) selected for the client terminal and transmitthe selected data segment through the server communication unit 820 tothe client terminal.

The server controller 810 may send a plurality of data segments to theclient terminal even when the client terminal has sent a single requestfor data. This may diminish the number of requests that the clientterminal is sends to the server and therefore, may also diminish theload on the client terminal, streaming server, and network due totransmission requests for streaming data.

The server controller 810 may further include the network monitoringunit 830. The network monitoring unit 830 may monitor the networkenvironment between the streaming server 800 and the client terminal.For example, the network monitoring unit 830 may calculate the currenttransmission rate between the streaming server and the client terminalbased on the size of the data segment transmitted to the client terminaland the time taken to transmit the data segment to the client terminal.

The streaming server 800 may further include a network logger (notshown). The network logger may produce, store, and manage logscontaining records regarding the times of transmission and recommendeddata. The network monitoring unit 830 may also determine the networkenvironment between the streaming server 800 and the client terminalbased on the log information from the network logger.

Further, the server controller 810 may determine the state (e.g.,playing timing, duration, resolution, or sound quality of multimediadata) of a data segment to be transmitted to the client terminal basedon at least one of the total capacity or remaining capacity of thememory of the client terminal or the maximum resolution at which theclient terminal may execute.

Further, the server controller 810 may determine the state of the datasegment to be transmitted to the client terminal depending on whetherthe connection means between the streaming server 800 and the clientterminal is wired communication or wireless communication. For example,the server controller 810 may determine the state of the data segmentbased on whether the client terminal is using a paid network offeredfrom a carrier (e.g., KT, SKT, or LGT) or is in linkage with Wi-Fi. Whenthe client terminal is in use of a carrier's paid network, the streamingserver 800 may transmit data segments with a playable resolution to theclient terminal without delay or buffering even at a transmission ratelower than the parameter value (e.g., data transmission rate) regardingthe network environment between the streaming server and the clientterminal, or may send data segments with the lowest resolution to theclient terminal regardless of the conditions of the network environment.

FIG. 9 is a block diagram illustrating a client terminal according to anembodiment of the disclosure.

Referring to FIG. 9, a client terminal 900 includes a terminalcontroller 910 (or at least one processor) and a terminal communicationunit 920 (or communicator or transceiver).

The terminal controller 910 includes an interface unit 930, a playingunit 940, and a network monitoring unit 950. The terminal controller 910may control or manage software or hardware executed on the clientterminal 900 or execute at least one command for executing the functionsof the client terminal 900. The terminal controller 910 may transmit, tothe streaming server, information, e.g., HTTP information, aboutmultimedia data to be played through an HTTP request.

The terminal communication unit 920 may receive multimedia data or adata segment from the streaming server.

The terminal controller 910 may calculate a parameter value regardingthe network environment between the streaming server and the clientterminal 900 when the multimedia data or data segment is transmitted tothe terminal communication unit 920, and send a request for at least onemultimedia data item, that it is to receive, to the streaming serverbased on the calculated parameter value. The terminal communication unit920 may further receive multimedia data or a data segment according tothe request from the streaming server.

The playing unit 940 plays media (e.g., video) using data segments(e.g., video data segments). The playing unit 940 may include a decoderand an application buffer.

The client terminal 900 may further include a decoder unit (not shown).The decoder unit may decode the data segment received from the streamingserver so that the data segment is converted to be output on an actualscreen.

The client terminal 900 may further include a reordering unit (notshown). The reordering unit may combine, in order, data segments (e.g.,video data segments) received from multiple wireless networks andtransmit the combined data segments to the application buffer of themedia player.

The network monitoring unit 950 may monitor the network environmentbased on the difference between when the multimedia data is sent fromthe streaming server and the time when the multimedia data is receivedby the client terminal 900, and the network monitoring unit 950 may senda request for a different data segment to the streaming server based ona result of monitoring the network environment.

Additionally, the network monitoring unit 950 may determine the networkenvironment based on whether a delay or buffering occurs in playing themultimedia data requested by the user to be played. Further, the clientterminal 900 may send a request for a different data segment to thestreaming server based on whether the play of the multimedia data isdelayed. Further, where the play of the multimedia data is delayed, theclient terminal 900 may produce an alarm signal to be displayed to theuser through the UI. For example, the alarm signal may include a warningindicating that it is requested to play multimedia data with a higherresolution as compared with the network environment and thus, a delayoccurs or a guide to request that multimedia data with a lowerresolution be played.

The client terminal 900 may further include a display unit (not shown)that receives decoding data from the decoder of the playing unit andoutputs on an actual screen.

FIG. 10 is a block diagram illustrating a streaming system according toan embodiment of the disclosure.

Referring to FIGS. 8 to 10, according to the present disclosure, astreaming system 1000 providing HTTP adaptive streaming includes astreaming server 1010 and a client terminal 1020. The streaming server1010 includes a server communication unit 1011 (or communicator ortransceiver) and a server controller 1012 (or at least one processor).The client terminal 1020 includes a terminal communication unit 1021 (orcommunicator or transceiver) and a terminal controller 1022 (or at leastone processor).

The streaming server 1010 may be the streaming server 800 of FIG. 8, andthe client terminal 1020 may be the client terminal 900 of FIG. 9.However, embodiments of the present disclosure are not limited thereto.

As an example, where the streaming server 1010 and the client terminal1020 in the streaming system 1000, each, includes a network monitoringunit for monitoring the network environment as shown in FIGS. 8 and 9,the streaming system 1000 may overall be more loaded but with the higheraccuracy associated with monitoring the network environment between thestreaming server 1010 and the client terminal 1020.

As another example, where a network monitoring unit for monitoring thenetwork environment is included only in the server controller 1012 ofthe streaming server 1010, this may advantageously reduce the load onthe client terminal 1020 which is typically subject to limitedresources.

The above-described operations may be realized by equipping a memorydevice retaining their corresponding codes in any component of theserver or terminal device of the communication system. That is, thecontroller in the server or terminal device may execute theabove-described operations by reading and running the program codesstored in the memory device by at least one processor or centralprocessing unit (CPU).

As described herein, various components or modules in the server orterminal device may be operated using a hardware circuit, e.g., acomplementary metal oxide semiconductor-based logic circuit, firmware,software, and/or using a hardware circuit such as a combination ofhardware, firmware, and/or software embedded in a non-transitory,machine-readable medium. As an example, various electric structures andmethods may be executed using electric circuits such as transistors,logic gates, or ASICs.

As is apparent from the foregoing description, according to the presentdisclosure, the server may monitor the network environment and providestreaming data that adaptively varies depending on the networkenvironment without overloading the client terminal.

The present disclosure may monitor the network environment based on thedifference between the start time of data transmission and the arrivaltime of an ACK and thus, become able to provide terminal-customized HTTPstreaming services in a more precise way.

While the disclosure has been shown and described with reference tovarious embodiments thereof, it will be understood by those skilled inthe art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the disclosure as definedby the appended claims and their equivalents.

What is claimed is:
 1. A method for providing, by a streaming server,hypertext transfer protocol (HTTP) adaptive streaming to a terminal, themethod comprising: transmitting a first multimedia data to the terminal;monitoring a network environment between the streaming server and theterminal when the first multimedia data is transmitted to the terminal;and pushing at least one second multimedia data based on the monitorednetwork environment.
 2. The method of claim 1, wherein the pushing ofthe at least one second multimedia data comprises: transmitting the atleast one second multimedia data to the terminal regardless of whetherthere is a request from the terminal after the streaming servertransmits the first multimedia data to the terminal.
 3. The method ofclaim 2, wherein the pushing of the at least one second multimedia datafurther comprises: transmitting the at least one second multimedia datato the terminal at predetermined times.
 4. The method of claim 3,wherein the predetermined times are determined based on at least one ofthe first multimedia data item, the number of the at least one secondmultimedia data item, or a playing duration.
 5. The method of claim 1,wherein the network environment indicates a transmission rate betweenthe streaming server and the terminal, and wherein the monitoring of thenetwork environment comprises calculating the transmission rate based ona difference between a time of starting transmission of the firstmultimedia data and a time of ending the transmission of the firstmultimedia data item.
 6. The method of claim 5, wherein the differenceis determined as a difference between the time of starting thetransmission of the first multimedia data and a time of receiving anacknowledgment (ACK) signal for the first multimedia data transmitted.7. The method of claim 1, wherein the first multimedia data includes atleast one of video data or audio data split into segments, and whereinthe pushing of the at least one second multimedia data comprisestransmitting the at least one second multimedia data with a resolutiondifferent from a resolution of the first multimedia data item.
 8. Themethod of claim 7, wherein the pushing of the at least one secondmultimedia data further comprises transmitting the at least one secondmultimedia data with a resolution lower than the resolution of the firstmultimedia data when the calculated transmission rate is lower than aprior transmission rate.
 9. The method of claim 7, wherein the pushingof the at least one second multimedia data further comprisestransmitting the at least one second multimedia data with a resolutionlower than the resolution of the first multimedia data when thecalculated transmission rate is lower than a transmission ratecorresponding to the resolution of the first multimedia data item.
 10. Astreaming server configured to provide HTTP adaptive streaming to aterminal, the streaming server comprising: a transceiver; and aprocessor configured to: control the transceiver to transmit a firstmultimedia data to the terminal, monitor a network environment betweenthe streaming server and the terminal when the first multimedia data istransmitted to the terminal, and control the transceiver to push atleast one second multimedia data based on the network environmentmonitored.
 11. The streaming server of claim 10, wherein, to push the atleast one second multimedia data item, the processor is furtherconfigured to: control the server communicator to transmit the at leastone second multimedia data to the terminal regardless of whether thereis a request from the terminal after the streaming server transmits thefirst multimedia data to the terminal.
 12. The streaming server of claim11, wherein the processor is further configured to control the servercommunicator to transmit the at least one second multimedia data to theterminal at predetermined times.
 13. The streaming server of claim 12,wherein the predetermined times are determined based on at least one ofthe first multimedia data item, the number of the at least one secondmultimedia data item, or a playing duration.
 14. The streaming server ofclaim 10, wherein the network environment indicates a transmission ratebetween the streaming server and the terminal, and wherein the processoris further configured to calculate the transmission rate based on adifference between a time of starting transmission of the firstmultimedia data and a time of ending the transmission of the firstmultimedia data item.
 15. The streaming server of claim 14, wherein thedifference is determined as a difference between the time of startingthe transmission of the first multimedia data and a time of receiving anacknowledgment (ACK) signal for the first multimedia data transmitted.16. The streaming server of claim 10, wherein the first multimedia dataincludes at least one of video data or audio data split into segments,and wherein the processor is further configured to control to push theat least one second multimedia data with a resolution different from aresolution of the first multimedia data item.
 17. The streaming serverof claim 16, wherein the processor is further configured to control topush the at least one second multimedia data with a resolution lowerthan the resolution of the first multimedia data when the calculatedtransmission rate is lower than a prior transmission rate.
 18. Thestreaming server of claim 16, wherein the processor is furtherconfigured to control to push the at least one second multimedia datawith a resolution lower than the resolution of the first multimedia datawhen the calculated transmission rate is lower than a transmission ratecorresponding to the resolution of the first multimedia data item.
 19. Aterminal configured to receive hypertext transfer protocol (HTTP)adaptive streaming from a streaming server, the terminal comprising: atransceiver; and a processor configured to: control the transceiver toreceive a first multimedia data from the streaming server, calculate aparameter value regarding a network environment between the streamingserver and the terminal when the first multimedia data is transmitted tothe terminal communicator, and control the transceiver to transmit arequest for at least one second multimedia data to be received bypushing to the streaming server based on the calculated parameter value.20. The terminal of claim 19, wherein the parameter value is calculatedbased on a difference between a time when the first multimedia data istransmitted from the streaming server and a time when the firstmultimedia data is received by the terminal.